fix(omlx): keep role thinking effort on discovered models - #4631
Conversation
There was a problem hiding this comment.
Adversarial red-team review at exact head d4ea42200f088620477be87eebabd5253f69d7b8 (base 8711c555da490ad36475f94368bcdd8d8f01048e, one commit, 9 files, +129/−4). Reviewed against current dev f3726ef05e507c9e0123fadbe17af833d1b1e7e3.
Verdict: MERGE_READY
Focused local verification (detached read-only worktree at d4ea422, review branch untouched)
bun test packages/coding-agent/test/omlx-discovery.test.ts \
packages/coding-agent/test/model-profiles-catalog.test.ts \
packages/ai/test/openai-completions-compat.test.ts \
packages/ai/test/model-thinking.test.ts \
packages/coding-agent/test/model-registry.test.ts
→ 365 pass / 0 fail / 4713 expect() calls
Notes on the run: the temp worktree needed bun --cwd=packages/natives run build (fresh checkout ships no addon; the global copies carry an older __piNativesV sentinel). Three model-registry.test.ts cases initially failed because this host exports ANTHROPIC_BASE_URL/OPENAI_BASE_URL pointing at a proxy — pre-existing test/env interaction, reproduces identically without the PR patch; after env -u of those vars all 365 pass. Exact-head CI run 32026121902 is green on all substantive affected-path, package, and evidence checks; only the needs-human contract jobs are red (PR body carries the verdict request), which this review addresses.
Findings (no blockers)
- Bug confirmed real at base. At 8711c55, discovery (
model-registry.ts:3042-3073) synthesized oMLX rows withreasoning: referenceModel?.reasoning ?? falseand hardcodedsupportsReasoningEffort: false, somacos-omlx-*role suffixes (:low/:medium/:highfrom #4607'smodel-profiles.ts:103-135) were clamped away before request generation. The patch fixes both sides. - Test allowlist matches the #4607 profiles exactly.
macosOmlxModelsinmodel-profiles-catalog.test.ts:688-693enumerates the same five ids the five built-in profiles reference (Qwen3.6-35B-A3B-{4bit,8bit,bf16},Qwen3.8-27B-Abliterated-MLX-{4bit,6bit}) — verified againstmodel-profiles.ts:103-135at base. It is test-harness-only (selector validation); runtime discovery still accepts any served oMLX id, so non-Qwen weights served via oMLX are not excluded, they just may not honor effort. - Effort normalization and omission semantics are correct.
getSupportedEfforts→ explicitthinking{min: low, max: high, default: medium}(expandEffortRange yields low/medium/high);clampThinkingLevelForModelclamps xhigh/max → high and minimal → low per the floor rule. Catalog-refresh path (applyGeneratedModelPolicies+inferFallbackEfforts/inferDefaultEffortinmodel-thinking.ts) is forced-consistent for oMLX regardless of what/v1/modelsreports. On the wire (openai-completions.ts:1400-1407),enable_thinkingis always emitted (false when reasoning is off/disabled — identical to base behavior), andreasoning_effortis added only when thinking is enabled and an effort was requested; no stale/minimal effort leaks into disabled turns. oMLX defines noreasoningEffortMap, so low/medium/high pass through verbatim. - Non-oMLX behavior unchanged. Every new provider-scoped branch in
model-thinking.ts,openai-compat.ts, andmodel-registry.tsis gated onprovider === "omlx"(theapplyGeneratedModelPolicyone additionally onapi === "openai-completions").git grepat head shows no bundled model or provider config outside oMLX usesthinkingFormat: "qwen-chat-template", so thebuildParamschange is scoped to oMLX plus any user who explicitly sets that format — for whom addingreasoning_effortonly when effort is present is a strict improvement, not a regression.github-copilotgate, zai/qwen/openrouter branches untouched. - Duplication is deliberate, not drift. Discovery-time defaults (
model-registry.ts/mapModelinopenai-compat.ts) and catalog-refresh enforcement (applyGeneratedModelPolicies) encode the same capability tuple; the refresh path is the safety net if a models.dev refresh rewrites oMLX rows. Values are identical across all three sites (reasoning true, low..high, medium default, qwen-chat-template, reasoning_content, no store, no developer role). - Current-dev interaction: none. f3726ef changes only
workflow-gate-broker.ts, its two test files, and a changelog — disjoint from every file this PR touches. Clean merge; no semantic overlap in thinking/model-registry paths. - Security/privacy: no new secret handling, logging, or URL surface; discovery error redaction (
redactDiscoveryUrl) unchanged; the oMLX token path is untouched.
Residual risk (accepted, non-blocking)
- No live oMLX dogfood was run (Linux host; oMLX is macOS-only). The patch assumes the served endpoint accepts
chat_template_kwargs.reasoning_effort. #4607 already committed this transport tochat_template_kwargsforenable_thinking, and the Qwen3 chat template documents the effort kwarg, so the assumption is well-grounded; a backend that ignores unknown kwargs degrades gracefully to #4607 behavior. - No explicit test for the
enable_thinking: false+ effort-present combination, but the code path structurally omitsreasoning_effortthere and existing tests cover the enabled path end-to-end.
Merge was not performed; this is review-only per the assignment.
—
[repo owner's gaebal-gajae (clawdbot) 🦞]
Discovered oMLX rows were synthesized as non-reasoning OpenAI models, so macos-omlx-* :low/:medium/:high suffixes clamped off before the wire. Treat oMLX catalog entries as reasoning models and send chat_template_kwargs.reasoning_effort with enable_thinking. Lore-id: 0m1xef47 Constraint: oMLX accepts reasoning_effort via chat_template_kwargs, not top-level OpenAI effort alone Rejected: drop preset effort suffixes | makes the one-model role contract a lie Confidence: high Scope-risk: narrow Reversibility: easy Tested: bun test packages/coding-agent/test/omlx-discovery.test.ts packages/coding-agent/test/model-profiles-catalog.test.ts packages/ai/test/openai-completions-compat.test.ts Not-tested: live oMLX Qwen template applying every effort literal
d4ea422 to
4f70373
Compare
Yeachan-Heo
left a comment
There was a problem hiding this comment.
Exact-head approval at rebased head 4f70373.
Re-verification of review 4953113614 (MERGE_READY at d4ea422): the rebase onto dev f3726ef is content-identical to the approved patch (zero-line delta excluding index hashes); re-ran locally at 4f70373 — bun test omlx-discovery/model-profiles-catalog/openai-completions-compat/model-thinking/model-registry: 365 pass / 0 fail; bun --cwd=packages/ai run check: pass; bun --cwd=packages/coding-agent run check: pass; fast gate (verify-gjc-state-writers --fail): pass; base f3726ef is ancestor; PR files disjoint from dev f3726ef delta (workflow-gate-broker only). Canonical diff digest unchanged: 53f2e0aa18fa88a404796b827db18afe6755e257b8dfb3b0b5d41ecc766e1c64.
No merge performed yet; this approval enables the exact-head contract transition.
—
[repo owner's gaebal-gajae (clawdbot) 🦞]
Yeachan-Heo
left a comment
There was a problem hiding this comment.
Contract transition record: verdict line updated needs-human → merge-approved (digest unchanged 53f2e0aa18fa88a404796b827db18afe6755e257b8dfb3b0b5d41ecc766e1c64, content-identical rebase), fresh exact-head approval 4953649002 at 4f70373. This comment re-triggers the review-event contract run against the current body.
—
[repo owner's gaebal-gajae (clawdbot) 🦞]
…4643) Dev CI shard-6 (run 32059470791, job 95479277528) failed both sdk-host-steer-integration tests at the harness transport's fixed 100 x Bun.sleep(1) poll with "host did not respond", while the same head passed on rerun (job 95485461325): a wall-clock poll bound, not a host response regression from #4631/#4613. Replace the poll loop with a per-emission promise handshake resolved by the response frame correlated to the live request id, fenced against stale/late frames (recorded, never resolving a later await), with a bounded safety timeout instead of a hang. Transport now also supports an injectable response delay for race-contract tests: delayed response beyond the old 100ms bound, timeout then fenced late response, and sequential control isolation. Durable replay/restart, no-redispatch, and undefined-session-file state-root contracts are preserved verbatim. Sandbox-only unrelated evidence recorded in the PR: tools/lsp-regressions C#-LSP/tlaplus tests fail under the run-bun-test-files sandboxed HOME at this head both with and without this patch (environment-sensitive, not absorbed in this lane). Closes #4641 Lore-id: 4641-steer-sync Constraint: no blind sleep inflation; handshake must be tied to actual host response Constraint: keep synchronous delivery semantics of the memory transport Rejected: raising the poll budget to e.g. 500 iterations | still wall-clock dependent under fresh-process load Rejected: unbounded await on response promise | a lost response would hang the file to the harness timeout Tested: focused file 3x (4/0), 3x fresh-process sandboxed (4/0), adjacent SDK suites 105/0, package check green Not-tested: none; CI shard-6 rerun on the PR head Confidence: high Scope-risk: narrow Reversibility: trivial Co-authored-by: Yeachan Heo <yeachan.heo@gmail.com> Co-authored-by: probe <re2rar@gmail.com>
…o#4631) Discovered oMLX rows were synthesized as non-reasoning OpenAI models, so macos-omlx-* :low/:medium/:high suffixes clamped off before the wire. Treat oMLX catalog entries as reasoning models and send chat_template_kwargs.reasoning_effort with enable_thinking. Lore-id: 0m1xef47 Constraint: oMLX accepts reasoning_effort via chat_template_kwargs, not top-level OpenAI effort alone Rejected: drop preset effort suffixes | makes the one-model role contract a lie Confidence: high Scope-risk: narrow Reversibility: easy Tested: bun test packages/coding-agent/test/omlx-discovery.test.ts packages/coding-agent/test/model-profiles-catalog.test.ts packages/ai/test/openai-completions-compat.test.ts Not-tested: live oMLX Qwen template applying every effort literal Co-authored-by: twoimo <twoimo@twoimoui-MacBookPro.local> (cherry picked from commit 80a0c09)
…eachan-Heo#4643) Dev CI shard-6 (run 32059470791, job 95479277528) failed both sdk-host-steer-integration tests at the harness transport's fixed 100 x Bun.sleep(1) poll with "host did not respond", while the same head passed on rerun (job 95485461325): a wall-clock poll bound, not a host response regression from Yeachan-Heo#4631/Yeachan-Heo#4613. Replace the poll loop with a per-emission promise handshake resolved by the response frame correlated to the live request id, fenced against stale/late frames (recorded, never resolving a later await), with a bounded safety timeout instead of a hang. Transport now also supports an injectable response delay for race-contract tests: delayed response beyond the old 100ms bound, timeout then fenced late response, and sequential control isolation. Durable replay/restart, no-redispatch, and undefined-session-file state-root contracts are preserved verbatim. Sandbox-only unrelated evidence recorded in the PR: tools/lsp-regressions C#-LSP/tlaplus tests fail under the run-bun-test-files sandboxed HOME at this head both with and without this patch (environment-sensitive, not absorbed in this lane). Closes Yeachan-Heo#4641 Lore-id: 4641-steer-sync Constraint: no blind sleep inflation; handshake must be tied to actual host response Constraint: keep synchronous delivery semantics of the memory transport Rejected: raising the poll budget to e.g. 500 iterations | still wall-clock dependent under fresh-process load Rejected: unbounded await on response promise | a lost response would hang the file to the harness timeout Tested: focused file 3x (4/0), 3x fresh-process sandboxed (4/0), adjacent SDK suites 105/0, package check green Not-tested: none; CI shard-6 rerun on the PR head Confidence: high Scope-risk: narrow Reversibility: trivial Co-authored-by: Yeachan Heo <yeachan.heo@gmail.com> Co-authored-by: probe <re2rar@gmail.com> (cherry picked from commit b72f125)
What
Discovered oMLX models now keep thinking metadata so
macos-omlx-*role suffixes (:low/:medium/:high) survive clamp and reach the local server.Wire format:
chat_template_kwargs.enable_thinkingpluschat_template_kwargs.reasoning_effort, matching oMLX Qwen templates.Catalog tests now allowlist the five served oMLX ids instead of treating every
omlx/<id>as valid.Rebased onto current
dev(f3726ef05, #4572) after maintainer MERGE_READY ond4ea42200.Why
#4607 advertised one-model / thinking-effort role mappings to avoid oMLX weight swap. Discovery synthesized generic OpenAI rows with
reasoning: falseandsupportsReasoningEffort: false, so every suffix was stripped before the request. The presets were a silent no-op.Follow-up to #4607 after post-merge architect review.
Testing
bun test packages/coding-agent/test/omlx-discovery.test.tsbun test packages/coding-agent/test/model-profiles-catalog.test.tsbun test packages/ai/test/openai-completions-compat.test.tsbun --cwd=packages/ai run checkbun --cwd=packages/coding-agent run checkGJC verdict
devbun checkpasses